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ABSTRACT:  The  MITRE  Corporation  in  Bedford,  MA  is  executing  a  small  research  project  entitled  “Capturing 
Behavioral  Influences  in  Synthetic  C2.  ”  This  project  is  being  sponsored  by  the  Air  Force  Electronic  Systems 
Center  (ESC)  and  began  in  November  of  2001.  At  the  previous  SIW  we  presented  the  project  plan.  This  paper 
presents  initial  findings  from  the  project  based  on  preliminary  prototyping  efforts  and  a  review  of  related  work  in 
the  community. 

We  originally  set  out  to  “start  simple  ”  by  modeling  a  single  C2  operator  in  the  Joint  Surveillance  and  Target  Attack 
Radar  System  (JSTARS)  mission  area;  however,  we  found  it  both  undesirable  and  infeasible  to  single  out  a  single 
operator  since  C2  at  its  core  is  collaborative  teamwork.  C2  modeling  efforts  need  to  focus  on  team  or  unit-level 
models.  To  make  better  use  of  limited  available  behavior  data,  C2  behavior  modeling  efforts  must  expand  on 
existing  information  processing  models  and  address  specific  taxonomies  of  C2  user  tasks.  Hybrid  models  may  be 
necessary  to  bring  the  best  of  multiple  modeling  approaches  to  bear  on  the  complex  nature  of  C2  team/unit 
modeling.  Guidelines  are  needed  to  (1)  better  define  appropriate  levels  of  detail/investment  in,  and  (2)  provide 
validation  approaches  for,  behavior  modeling  across  the  different  C2  application  areas.  Finally,  in  efforts  to 
interface  interactive  human  behavior  models  with  battle  simulations,  it  is  necessary  to  understand  and  refine  the 
“hooks  ”  that  enable  the  behavior  models  to  appropriately  impact  the  simulated  battle. 


1,  Introduction 

This  paper  provides  initial  findings  from,  and  a 
proposed  path  forward  for,  a  behavior  modeling  project 
effort  that  started  in  November  2001.  This  project  is 
entitled  Capturing  Behavioral  Influences  in  Synthetic 
C2  and  is  being  executed  by  the  MITRE  Corporation  in 
support  of  the  Air  Force  Electronic  Systems  Center 
(ESC).  The  purpose  of  the  project  is  to  explore 
application  of  emerging  behavioral  modeling  techniques 
within  military  simulations  to  better  capture  the 
cognitive  and  organizational  factors  that  influence  C2 
operations  [Flournoy,  2002]. 

This  paper  begins  with  a  background  section  that 
describes  the  motivation  behind  this  research  project 
and  the  potential  for  it  and  other  similar  efforts  to 
positively  impact  simulation-based  C2  system 
development  applications.  Then,  a  brief  overview  of 


progress  on  the  project  in  FY02  is  provided  in  Section 
3.  This  leads  into  sections  that  detail  our  lessons 
learned  on  the  two  facets  of  the  project:  (1)  developing 
the  JSTARS  Operator  Surrogate  Human  (JOSH) 
(Section  4),  and  (2)  exploring  the  potential  for 
connecting  JOSH  to  a  battle  simulation  to  “play”  a 
JSTARS  mission  within  a  larger  battle  context  (Section 
5).  Based  on  these  lessons  learned,  we  outline  a  path 
forward  for  the  work  in  Section  6.  Finally  in  Section  7 
we  state  the  potential  impact  of  this  work  in 
collaboration  with  government  and  industry  partner 
projects  as  we  each  strive  for  comprehensive  yet  cost- 
effective  approaches  to  building  and  applying  behavior¬ 
centric  models  of  C2. 

2,  Background 

We  began  this  research  thrust  in  FY02,  citing  a  National 
Research  Council  (NRC)  study  commonly  referred  to  as 
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the  “Green  Book”  (pictured 
below)  that  put  forth 
recommendations  to  improve 

human  behavior  representation  in 
military  simulations  [NRC, 

1998].  Indeed  on  a  local  level  at 
ESC  we  recognized  that  the 
representation  of  decision¬ 
making  behaviors  in  our  C2 
simulations  did  not  begin  to 
match  our  well-developed 
models  of  the  more  physical  Figure  1.  The 
aspects  of  C2  (for  example,  "Green  Book" 
communication  channels  and 
sensor  performance).  While  other  DoD  research 

organizations  have  invested  heavily  in  behavior 
modeling  for  many  years — including  AFRL,  the  Army 
Modeling  and  Simulation  Office  (AMSO),  the  Defense 
Modeling  and  Simulation  Office  (DMSO),  and  the 
Office  of  Naval  Research  (ONR) — our  MOIE  effort  in 
FY02  marked  ESC’s  initial  entry  in  the  field. 

Continued  ESC  investment  in  behavior  modeling 
approaches  is  now  critical  with  the  emergence  of  new 
C2  initiatives  such  as  the  Multi-mission  C2  Aircraft 
(MC2A)  and  the  Air  Operations  Center  (AOC)  Time 
Critical  Targeting  (TCT)  Cell.  These  initiatives  rely  on 
teams  of  C2  operators  dealing  with  increased 
information  flow  under  reduced  manpower  and 
footprint  conditions.  Modeling  the  behavior  of  such 
teams  is  key  to  determining  how  advanced  systems  and 
operational  concepts  can  best  be  employed  to  assist 
them. 

Figure  2  illustrates  a  process  for  leveraging  behavior 
models  to  augment  C2  behaviors  in  battle  simulations. 
First,  a  task  analysis  is  undertaken  to  determine 
roles/responsibilities  and  the  associated  behaviors  of  the 
subject  warfighter  or  team  of  warfighters.  Then,  these 
behaviors  are  entered  into  a  behavior  modeling  tool  to 
produce  an  standalone  executable  behavior  model. 
Finally,  this  executable  behavior  model  is  interfaced  to 
a  battle  simulation  so  that  it  receives  situational  updates 
from  the  simulation  and  responds  with  decisions  that 
effect  the  progress  of  the  battle  in  the  simulation.  Note 
the  dual-headed  arrows  between  each  step  in  the 
process:  although  the  desired  flow  of  the  process 
proceeds  from  left  to  right  in  the  diagram,  in  practice 
the  process  is  iterative  in  nature.  The  capabilities  of  the 
behavior  modeling  tool  dictate  to  some  extent  the  level 
and  focus  of  the  task  analysis.  Similarly,  the  nature  of 
the  selected  battle  simulation  influences  the  design  of 
the  executable  behavior  model. 


Figure  2.  A  Process  for  Capturing  Behavioral 
Influences  in  Synthetic  C2. 


3,  Overview  of  FY02  Progress 

We  began  the  research  effort  last  year  by  first  assessing 
the  C2  modeling  potential  of  available  behavior 
modeling  tools  and  battle  simulations.  Based  on  these 
findings  we  developed  an  initial  prototype  of  the  Joint 
STARS  Operator  Surrogate  Human  (JOSH)  C2 
behavior  model  using  the  AFRL-sponsored  Crew 
Automation  Requirements  Testbed  (CART)  modeling 
tool  [Martin,  2000].  We  selected  CART  over  other 
behavior  modeling  tools  for  our  initial  prototyping 
efforts  because  of  these  perceived  capabilities: 

•  Task-network  approach  and  mature  graphical  user 
interface  (GUI)  for  creating  an  initial  model  from 
task  analysis  results, 

•  Goals  &  ability  to  encode  overarching  effects, 

•  Workload  estimation  based  on  validated  research, 

•  HLA  interface,  and 

•  Successful  history  on  AFRL  project  work. 

While  developing  the  JOSH  C2  behavior  model  using 
CART,  we  selected  and  installed  a  set  of  applications  to 
form  the  basis  for  a  simulated  battle  testbed  for  CART. 
For  our  primary  battle  simulation  we  considered  the 
Extended  Air  Defense  Simulation  (EADSIM)  and  the 
Joint  Interim  Mission  Model  (JIMM).  Both  of  these 
mission-level  simulations  have  achieved  a  high  level  of 
acceptance  at  ESC  across  a  variety  of  analysis  and 
experimentation  projects.  We  selected  EADSIM  for 
our  initial  research  efforts  on  the  strength  of  (1)  its 
strong  reputation  for  modeling  the  physical  aspects  of 
C2,  and  (2)  its  straightforward  GUI  for  facilitating 
battle  scenario  development. 

We  federated  the  Joint  STARS  Transportable  Emulator 
(JSTE)  with  EADSIM  in  order  to  provide  (I)  a  JSTARS 
human-in-the-loop  console  and  (2)  JSTARS  sensor 
output  information  for  the  testbed.  Finally,  we 
federated  the  Integrated  Target  Environment  Simulation 
Tool  (ITEST)  with  EADSIM  and  the  JSTE  to  provide 
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additional  ground  vehicle  traffic  if  necessary  to  create  a 
realistic  environment  for  JOSH. 


4,  Lessons  Learned  Using  CART  to  Build 
JOSH 

Figure  3,  below,  provides  a  high-level  overview  of  our 
findings  from  our  initial  FY02  efforts,  and  based  on 
these  findings,  potential  focus  areas  for  further  research 
in  FY03.  The  remaining  text  in  this  section  and  in 
Section  5  details  the  project  experiences  that  led  to 
these  findings.  Then  in  Section  6,  a  plan  is  presented 
for  addressing  some  of  the  proposed  focus  areas. 


What  We’ve  Learned  in  ‘02...  Where  We  Need  to  Go  in  ‘03. 
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Figure  3.  Overview  of  Initial  Findings  and  Focus 


Areas  for  Future  Research 


Our  tailored  Cognitive  Task  Analysis  (CTA)  techniques 
were  key  to  successfully  representing  operator  roles  and 
responsibilities  in  JOSH.  We  leveraged  CART’s  well- 
designed  user  interface  and  task-network  modeling 
approach  to  quickly  construct  a  first-order  version  of 
JOSH  and  proceed  through  several  evaluation  and 
refinement  spirals  of  JOSH. 

One  of  our  first  JOSH  spirals  involved  raising  the  level 
of  authority  of  the  roles  and  responsibilities  modeled  in 
JOSH  so  that  its  decisions  could  impact  events  in  a 
larger  simulated  battle  context  and  be  more  compatible 
with  existing  battlespace  environments  such  as 
EADSIM.  Initially  intended  to  be  focused  on  a  single 
position  within  the  operations  section  of  the  Joint 
STARS  aircraft,  we  recognized  a  need  to  expand  JOSH 
to  cover  the  effect  of  the  entire  Joint  STARS  operations 
team  on  the  battle  by  refocusing  on  the  Senior  Director 
(team  leader)  position  with  additional  tasks  included  to 
cover  many  of  the  other  team  members’  roles. 
Concurrent  with  our  first  year  efforts,  the  Human 
Effectiveness  Directorate  of  AFRL  began  using  CART 
to  model  Air  Operations  Center  (AOC)  activities.  For 
reasons  similar  to  ours  in  focusing  on  the  Senior 


Director  position  for  Joint  STARS,  AFRL  has  also 
chosen  to  focus  their  AOC  ISR  Management  model  on 
the  team  leader  position. 

Although  CART  and  the  other  behavior  modeling  tools 
we  examined  showed  much  promise,  none  alone  offered 
all  the  key  elements  necessary  to  bring  behavior 
modeling  technology  to  bear  on  the  C2  simulation  needs 
of  our  ESC  sponsors.  Each  of  the  tools  lacked  either  (1) 
the  cognitive  functionality  to  represent  key  Joint 
STARS  tasks  (for  example,  elevating  the  status  of  a 
nominated  track  to  a  time-critical  target),  or  (2)  the  user 
interface  maturity  necessary  for  cost-effective 
application  by  other  than  the  developers  of  the  tool.  In 
addition,  support  to  interfacing  these  tools  to  an  external 
simulated  battle,  and  thereby  realizing  their  potential  to 
enhance  a  variety  of  C2  simulation  use  cases,  was 
rudimentary  at  best. 

In  addition  to  experimenting  with  JOSH  in  the  lab,  we 
developed  several  technical  working  alliances  with 
related  government  and  industry  teams.  As  we  shared 
knowledge  with  these  teams  and  others  from  the 
growing  DoD  behavior  modeling  “community-at- 
large,”  we  found  that  the  above  challenges  we  identified 
last  year  were  echoed  frequently  by  others  as  crucial 
hurdles  to  overcome  via  additional  research  and 
development. 

5,  Lessons  Learned  Toward  Running 
JOSH  within  a  Simulated  Battle 

In  working  to  connect  our  EADSIM/JSTE/ITEST 
federation  to  the  CART -based  JOSH  model,  we 
identified  several  technical  hurdles  that  we  needed  to 
cross  to  achieve  a  working  interface  to  JOSH: 

•  Our  behavior  model  needed  to  focus  on  a  team 
leader  (management/decision  maker)  to  impact 
the  battle  in  EADSIM.  As  a  result  of  this  finding, 
as  discussed  in  Section  4,  we  refocused  our  JOSH 
model  on  the  JSTARS  Senior  Director  role; 
however,  that  committed  us  to  additional 
algorithmic  or  behavior  modeling  work  to  model 
the  information  inputs  of  the  team  members  to  the 
Senior  Director  to  feed  his  decisions. 

•  EADSIM  Version  Release  Issues.  The  planned 
release  of  Version  10  of  EADSIM,  which  contained 
the  capability  to  process  tracks  from  external 
sources  (e.g.,  JOSH),  was  delayed  until  we  were 
three-quarters  through  our  initial  year  of  research. 

•  CART  HLA  Interface  Limitations.  The  version 
of  CART  we  worked  with  was  limited  to  operating 
in  only  Real-time  Platform  Reference  (RPR) 
Federation  Object  Model  (FOM)  federations,  and 
implemented  only  a  small  part  of  this  FOM  in  its 
interface.  If  a  flexible-FOM  HLA  “front  end”  is 


implemented  in  CART  in  the  future,  this  would 
considerably  ease  attempts  to  federate  CART-based 
models  with  battle  simulations.  We  feel  this  is  a 
worthy  target  for  funding  in  the  future,  as  a  more 
flexible  data  interface  would  make  this  already 
promising  tool  even  more  attractive  to  a  broad  user- 
base  of  military  behavior  simulators. 

Because  of  the  challenges  we  faced  interfacing  our 
EADSIM/JSTE/ITEST  federation  with  JOSH,  we  began 
exploring  the  option  of  using  a  MITRE-developed  C2 
microworld  as  our  testbed  for  JOSH.  This  microworld 
was  initially  developed  under  the  name  “Friends  or 
Foes”  for  the  CAFC2S  project  Mental  Models  in 
Naturalistic  Decision  Making  [Bums,  2000].  This 
microworld  approach  provides  complete  access  to 
source  code;  and  with  that,  streamlines  many  of  the 
testbed  support  tasks  necessary  for  the  research  project. 
During  initial  risk-assessment  prototyping  efforts  with 
this  microworld  we  were  pleasantly  surprised  with  the 
leap  we  experienced  in  (1)  interface  flexibility,  (2)  ease 
of  incorporating  critical  scenario  development 
capabilities,  and  (3)  ease  of  developing  operator  cueing 
and  data  collection  capabilities  to  better  support  JOSH 
experimentation. 

6,  Path  Forward 

We  seek  to  extend  our  research  to  a  second  year  that 
will  address  key  challenges  in  C2  behavior  modeling  by 
pioneering  cost-effective  methods  for  applying  and  re¬ 
using  state-of-the-art  human  behavior  representation 
(HBR)  and  data  exchange  techniques.  The  work  will 
focus  on  the  extension  of  JOSH  to  incorporate  hybrid 
modeling  techniques.  We  will  experiment  with  these 
techniques  within  our  flexible  C2  microworld  battle 
environment.  By  following  this  approach  we  will  also 
learn  lessons  regarding  data  exchange  techniques  that 
can  bridge  human  behavior  modeling  tools  with  one 
another  and  with  battle  simulations  at  mntime.  We  will 
continue  to  draw  on  interdisciplinary  resources  across 
MITRE  and  technical  working  alliances  across  the 
research  community  by  examining  the  results  of  AOC 
Cognitive  Task  Analyses  (MITRE-Bedford)  and  AOC 
and  MC2A  modeling  efforts  (AFRL,  Lincoln 
Labs/Aptima)  for  our  research. 

The  text  below  in  Sections  6.1  and  6.2  details  our 
proposed  technical  approach  and  anticipated  products, 
respectively. 

6.1  Technical  Approach 

Here  is  a  detailed  outline  of  our  proposed  approach: 

•  Extend  JOSH  to  incorporate  hybrid  modeling 
techniqnes  that  better  address  C2  cognitive 


fnnctions  and  team  or  nnit  behavior. 

(1)  Examine  the  cost-effective  feasibility  of  several 
promising  models  that  were  identified  in  the  first 
year  of  this  effort  (e.g.,  ACT-R,  SOAR,  etc.). 

(2)  Model  key  cognitive  functions  with  selected 
tools  to  supplement  existing  behavior  modeling. 

(3)  Construct  data  exchange  “bridges”  to  link  the 
existing  surrogate  behavior  model  to  the  new 
cognitive  models. 

•  Expand  on  the  existing  C2  microworld  battle 
simnlation  to  serve  as  a  testbed  for  hybrid  model 
refinement  and  data  exchange  techniqnes. 

(1)  Evolve  the  previously  developed  C2  microworld 
(Friends  or  Foes)  to  interact  with  the  desired 
versions  of  JOSH. 

(2)  Establish  cueing  and  data  collection  in  the  C2 
microworld  to  support  collection  of  human 
operator  behaviors. 

(3)  Examine  protocols  to  standardize  data  exchange 
between  human  behavior  models  and 
microworlds/battle  simulations,  and  providing 
guidance  and  requirements  to  battle  simulation 
developers  regarding  the  “hooks”  where  HBR 
can  make  a  difference. 

•  Examine  the  cost-effectiveness  of  snpplementing 
the  cnrrent  Joint  STARS  software  snrrogate  to 
better  represent  generic  C2  (AOC,  MC2A,  and 
TCT)  modeling  efforts. 

(1)  Analyze  the  emergent  AOC  and  MC2A  human 
behavior  modeling  efforts  (e.g.,  AFRL,  Lincoln 
Labs/Aptima)  for  re-use  potential. 

(2)  Review  cognitive  task  analysis  efforts  from 
within  the  MITRE  Center  for  Air  Force  C2 
Systems  (CAFC2S)  for  consistent  taxonomy  and 
applicability. 

(3)  Examine  the  cost-effectiveness  of 
supplementing  the  existing  JOSH  with  a  more 
refined  AOC  model  that  better  represent  AOC 
cognitive  decision  making  tasks. 

6.2  Products 

Our  goal  will  be  to  provide  the  following  products  as  a 

result  of  next  years’  efforts: 

•  JOSH  v2  operating  in  Microworld  environment 

•  Papers  summarizing  lessons  learned  from: 

(1)  hybrid  modeling  investigation, 

(2)  Microworld-to-Behavior  Model  interface 
efforts,  and 

(3)  cost-effectiveness  analyses. 

7.  Impact  and  Technology  Transition 
Opportunities 


command  and  control. 


With  expertise  in  C2  systems  and  operations,  simulation 
interoperability,  and  human  performance  modeling,  the 
MITRE  team  is  looking  to  continue  to  bring  this 
interdisciplinary  team  to  bear  on  applications  of  human 
performance  modeling  methods  in  synthetic  C2. 
Coming  efforts  on  the  project  will  impact  these  three 
main  areas: 

•  Educated  investment  in  C2  behavior  modeling. 
By  targeting  key  challenges  faced  by  this  project 
last  year  and  echoed  by  our  industry  and 
government  partners,  this  effort  promotes  (1) 
increasing  technical  quality  and  (2)  reducing  the 
costs  associated  with  applying  behavior  models  in 
C2  simulation  efforts. 

•  Enhanced  realism  in  simulated  C2.  The  work  has 
potential  to  enhance  the  realism  of  simulated  battles 
at  key  human  perception  and  decision-making 
points.  These  can  realize  useful  metrics  in  the 
struggle  to  decrease  footprint  and  maintain 
effectiveness.  This  promises  to  improve  the  quality 
of  simulation  support  in  several  application  areas 
including  analysis  and  experimentation.  Programs 
likely  to  benefit  from  the  work  include 
MC2A/MC2C  and  TCT  Cell  efforts.  For  example, 
within  the  MC2A,  the  crew  may  be  called  upon  to 
perform  ground  and  air  surveillance  tasks,  cover 
various  ISR  management  tasks  (possibly  extending 
to  space-based  radar  and  UAV  control)  and  serve 
as  an  AOC  execution  cell.  Team  roles  and 
responsibilities  must  be  carefully  designed  for 
maximum  effectiveness  and  flexibility  to  meet 
these  lofty  objectives;  likewise,  computer  systems 
must  support  this  flexibility.  We  believe  the 
lessons  learned  from  this  MOIE  have  great 
potential  to  contribute  to  addressing  these  MC2A 
challenges.  From  a  mission-area  standpoint,  our 
growing  Joint  STARS  expertise  coupled  with 
AFRL’s  AOC  work  and  the  work  of  Aptima,  Inc. 
in  AW  ACS  team  design  form  a  powerful  base  for 
collaboration  toward  an  MC2A  analysis  capability. 

•  Influencing  the  state-of-the-practice  in  the 

community-at-large.  Having  established 

relationships  with  several  related  government  and 
industry  project  teams  last  year,  we  are  now  in  a 
position  to  share  findings,  provide 
recommendations,  and  influence  standards 
development. 

While  acknowledging  the  challenge  of  constructing 
models  that  capture  the  cognitive  complexities  of  team 
behavior,  we  have  made  substantial  progress  and  seek 
to  continue  our  focus  on  cost-effective  reuse  of  state-of- 
the-art  behavior  modeling  efforts  that  impact  synthetic 
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